<!DOCTYPE html>
<html lang="zh-CN" class="loading">

<head>
    <meta charset="UTF-8" />
    <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1" />
    <meta name="viewport" content="width=device-width, minimum-scale=1.0, maximum-scale=1.0, user-scalable=no">
    <title>一个简单的神经网络框架 - Skyone-Blog</title>
    <meta name="apple-mobile-web-app-capable" content="yes" />
    <meta name="apple-mobile-web-app-status-bar-style" content="black-translucent">
    <meta name="google" content="notranslate" />
    <meta name="keywords" content="Python,神经网络,"> 
    <meta name="description" content="Skyone技术博客,本文介绍了我自己写的一个简单的手写数字识别程序的原理"> 
    <meta name="author" content="王志伟"> 
    <link rel="alternative" href="atom.xml" title="Skyone-Blog" type="application/atom+xml"> 
    <link rel="icon" href="/img/favicon.png"> 
    
<link rel="stylesheet" href="//cdn.jsdelivr.net/npm/gitalk@1/dist/gitalk.css">

    
<link rel="stylesheet" href="/css/diaspora.css">

    <!-- 看板娘 -->
    <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/font-awesome/css/font-awesome.min.css">
    <script src="/live2d-widget/autoload.js"></script>
    <!-- MD5 -->
    <script src="/js/md5.min.js"></script>
    <!-- MathJax -->
    
        <script type="text/x-mathjax-config">
        (function () {
            let target = document.getElementsByClassName("content")[0];
            MathJax.Hub.Config({
                tex2jax: {
                    inlineMath:  [ ["$", "$"] ],
                    displayMath: [ ["$$","$$"] ],
                    skipTags: ['script', 'noscript', 'style', 'textarea', 'pre','code','a'],
                    ignoreClass:"class1"
                }
            });
            MathJax.Hub.Queue(["Typeset",MathJax.Hub,target]);
        })();
    </script>
<script src="https://polyfill.io/v3/polyfill.min.js?features=es6"></script>
<script id="MathJax-script" async src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>

    
<meta name="generator" content="Hexo 5.3.0"></head>

<body class="loading">
    <span id="config-title" style="display:none">Skyone-Blog</span>
    <div id="loader"></div>
    <div id="single">
    <div id="top" style="display: block;">
    <div class="bar" style="width: 0;"></div>
    <a class="iconfont icon-home image-icon" href="javascript:;" data-url="https://skyone-wzw.gitee.io"></a>
    <div title="播放/暂停" class="iconfont icon-play"></div>
    <h3 class="subtitle">一个简单的神经网络框架</h3>
    <div class="social">
        <div>
            <div class="share">
                <a title="获取二维码" class="iconfont icon-scan" href="javascript:;"></a>
            </div>
            <div id="qr"></div>
        </div>
    </div>
    <div class="scrollbar"></div>
</div>

    <div class="section">
        <div class="article">
    <div class='main'>
        <h1 class="title">一个简单的神经网络框架</h1>
        <div class="stuff">
            <span>十一月 14, 2020</span>
            
  <ul class="post-tags-list" itemprop="keywords"><li class="post-tags-list-item"><a class="post-tags-list-link" href="/tags/Python/" rel="tag">Python</a></li><li class="post-tags-list-item"><a class="post-tags-list-link" href="/tags/%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C/" rel="tag">神经网络</a></li></ul>


        </div>
        <div class="content markdown">
            <h2 id="简单介绍"><a href="#简单介绍" class="headerlink" title="简单介绍"></a>简单介绍</h2><p>这是一个基于python的最简单的神经网络框架，使用了<code>numpy</code>库，使用的是输入层 - 隐藏层 - 输出层的结构，每一层的节点数自可定义。原理图：</p>
<p><img src="https://tc.skyone.host/blog/post/%E4%B8%80%E4%B8%AA%E7%AE%80%E5%8D%95%E7%9A%84%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C%E6%A1%86%E6%9E%B6/%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C.png" alt="原理图"></p>
<p>激活函数是经典的：</p>
<p><img src="https://tc-1.oss-cn-hangzhou.aliyuncs.com/blog/post/%E4%B8%80%E4%B8%AA%E7%AE%80%E5%8D%95%E7%9A%84%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C%E6%A1%86%E6%9E%B6/%E6%BF%80%E6%B4%BB%E5%87%BD%E6%95%B0.png" alt="激活函数"></p>
<p>误差函数的斜率：</p>
<p><img src="https://tc-1.oss-cn-hangzhou.aliyuncs.com/blog/post/%E4%B8%80%E4%B8%AA%E7%AE%80%E5%8D%95%E7%9A%84%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C%E6%A1%86%E6%9E%B6/%E8%AF%AF%E5%B7%AE%E5%87%BD%E6%95%B0.png" alt="误差函数"></p>
<p>使用矩阵简化运算：</p>
<p><img src="https://tc-1.oss-cn-hangzhou.aliyuncs.com/blog/post/%E4%B8%80%E4%B8%AA%E7%AE%80%E5%8D%95%E7%9A%84%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C%E6%A1%86%E6%9E%B6/%E7%9F%A9%E9%98%B5%E8%BF%90%E7%AE%97.png" alt="矩阵运算"></p>
<p>代码很短、很简单，但效果却还不错。</p>
<p>输入层、隐藏层、输出层节点数分别784、100、10个，经过<code>MNIST手写数字数据集</code>的训练后，跑了10000个测试，手写数字识别准确率达到了95%左右。</p>
<p><img src="https://tc.skyone.host/blog/post/%E4%B8%80%E4%B8%AA%E7%AE%80%E5%8D%95%E7%9A%84%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C%E6%A1%86%E6%9E%B6/%E8%AF%86%E5%88%AB%E5%87%86%E7%A1%AE%E7%8E%87.png" alt="识别准确率"></p>
<h2 id="代码"><a href="#代码" class="headerlink" title="代码"></a>代码</h2><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">import</span> numpy</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">activate</span>(<span class="params">x</span>):</span></span><br><span class="line">    <span class="keyword">return</span> <span class="number">1</span> / (<span class="number">1</span> + numpy.exp(-x))</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">NeuralNetwork</span>:</span></span><br><span class="line">    <span class="function"><span class="keyword">def</span> <span class="title">__init__</span>(<span class="params">self, inputnodes, hiddennodes, outputnodes, learningrate=<span class="number">0.3</span></span>):</span></span><br><span class="line">        <span class="comment"># set number of nodes in each input, hidden, output layer</span></span><br><span class="line">        <span class="comment"># 分别是：输入层、隐藏层、输出层的节点数</span></span><br><span class="line">        self.inodes = inputnodes</span><br><span class="line">        self.hnodes = hiddennodes</span><br><span class="line">        self.onodes = outputnodes</span><br><span class="line"></span><br><span class="line">        <span class="comment"># learning rate</span></span><br><span class="line">        <span class="comment"># 学习速率</span></span><br><span class="line">        self.lr = learningrate</span><br><span class="line"></span><br><span class="line">        <span class="comment"># weights</span></span><br><span class="line">        <span class="comment"># 使用正态分布初始化权重</span></span><br><span class="line">        self.wih = numpy.random.normal(<span class="number">0.0</span>, <span class="built_in">pow</span>(self.inodes, -<span class="number">0.5</span>), (self.hnodes, self.inodes))</span><br><span class="line">        self.who = numpy.random.normal(<span class="number">0.0</span>, <span class="built_in">pow</span>(self.hnodes, -<span class="number">0.5</span>), (self.onodes, self.hnodes))</span><br><span class="line"></span><br><span class="line">    <span class="function"><span class="keyword">def</span> <span class="title">query</span>(<span class="params">self, inputs_list</span>):</span></span><br><span class="line">        <span class="comment"># 计算每个数字的概率</span></span><br><span class="line">        </span><br><span class="line">        <span class="comment"># 翻转矩阵</span></span><br><span class="line">        inputs = numpy.array(inputs_list, ndmin=<span class="number">2</span>).T</span><br><span class="line">		<span class="comment"># 输入进隐藏层的数据与权重相乘</span></span><br><span class="line">        hidden_inputs = numpy.dot(self.wih, inputs)</span><br><span class="line">        <span class="comment"># 激活函数</span></span><br><span class="line">        hidden_outputs = activate(hidden_inputs)</span><br><span class="line">		<span class="comment"># 输入进输出层的数据与权重相乘</span></span><br><span class="line">        final_inputs = numpy.dot(self.who, hidden_outputs)</span><br><span class="line">        <span class="comment"># 激活函数</span></span><br><span class="line">        final_outputs = activate(final_inputs)</span><br><span class="line"></span><br><span class="line">        <span class="keyword">return</span> final_outputs</span><br><span class="line"></span><br><span class="line">    <span class="function"><span class="keyword">def</span> <span class="title">train</span>(<span class="params">self, inputs_list, targets_list</span>):</span></span><br><span class="line">        <span class="comment"># 误差计算函数</span></span><br><span class="line">        inputs = numpy.array(inputs_list, ndmin=<span class="number">2</span>).T</span><br><span class="line">        targets = numpy.array(targets_list, ndmin=<span class="number">2</span>).T</span><br><span class="line"></span><br><span class="line">        hidden_inputs = numpy.dot(self.wih, inputs)</span><br><span class="line">        hidden_outputs = activate(hidden_inputs)</span><br><span class="line"></span><br><span class="line">        final_inputs = numpy.dot(self.who, hidden_outputs)</span><br><span class="line">        final_outputs = activate(final_inputs)</span><br><span class="line"></span><br><span class="line">        <span class="comment"># errors</span></span><br><span class="line">        <span class="comment"># 误差值</span></span><br><span class="line">        output_errors = targets - final_outputs</span><br><span class="line">        hidden_errors = numpy.dot(self.who.T, output_errors)</span><br><span class="line"></span><br><span class="line">        <span class="comment"># update the weights for the links between the hidden and output layer</span></span><br><span class="line">        <span class="comment"># 根据误差调整权重</span></span><br><span class="line">        self.who += self.lr * numpy.dot((output_errors * final_outputs * (<span class="number">1.0</span> - final_outputs)), numpy.transpose(hidden_outputs))</span><br><span class="line">        <span class="comment"># update the weights for the links between the hidden and output layer</span></span><br><span class="line">        <span class="comment"># 根据误差调整权重</span></span><br><span class="line">        self.wih += self.lr * numpy.dot((hidden_errors * hidden_outputs * (<span class="number">1.0</span> - hidden_outputs)), numpy.transpose(inputs))</span><br></pre></td></tr></table></figure>

            <!--[if lt IE 9]><script>document.createElement('audio');</script><![endif]-->
            <audio id="audio" loop="1" preload="auto" controls="controls" data-autoplay="false">
                <source type="audio/mpeg" src="">
            </audio>
            
                <ul id="audio-list" style="display:none">
                    
                        
                            <li title='0' data-url='https://tc.skyone.host/blog/music/%E6%B4%9B%E5%A4%A9%E4%BE%9D%20-%20%E5%A4%9C%E8%88%AA%E6%98%9F.flac'></li>
                        
                    
                        
                            <li title='1' data-url='https://tc.skyone.host/blog/music/%E6%B4%9B%E5%A4%A9%E4%BE%9D%20-%20%E8%8C%89%E8%8E%89%E8%8A%B1.mp3'></li>
                        
                    
                        
                            <li title='2' data-url='https://tc.skyone.host/blog/music/%E8%B5%A4%E7%BE%BD%20-%20%E4%B8%87%E5%8F%A4%E7%94%9F%E9%A6%99.flac'></li>
                        
                    
                        
                            <li title='3' data-url='https://tc.skyone.host/blog/music/%E6%B4%9B%E5%A4%A9%E4%BE%9D%20-%20%E4%B8%BA%E8%B0%81%E8%80%8C%E4%B8%BA.mp3'></li>
                        
                    
                        
                            <li title='4' data-url='https://tc.skyone.host/blog/music/%E4%B9%90%E6%AD%A3%E7%BB%AB%20-%20Faded.mp3'></li>
                        
                    
                        
                            <li title='5' data-url='https://tc.skyone.host/blog/music/bilibili2018%20-%20%E8%87%B3%E5%B0%8A%E9%A9%AC%E7%94%B2.flac'></li>
                        
                    
                </ul>
            
        </div>
        
    <div id='gitalk-container' class="comment link"
		data-enable='true'
        data-ae='true'
        data-ci='d5e197ef955b7f3268e5'
        data-cs='7d4feed7a179ad28943b0865b7970814073145ad'
        data-r='blog-gitalk'
        data-o='skyone-wzw'
        data-a='skyone-wzw'
        data-d='false'
    >查看评论</div>


    </div>
    
        <div class='side'>
			<ol class="toc"><li class="toc-item toc-level-2"><a class="toc-link" href="#%E7%AE%80%E5%8D%95%E4%BB%8B%E7%BB%8D"><span class="toc-number">1.</span> <span class="toc-text">简单介绍</span></a></li><li class="toc-item toc-level-2"><a class="toc-link" href="#%E4%BB%A3%E7%A0%81"><span class="toc-number">2.</span> <span class="toc-text">代码</span></a></li></ol>	
        </div>
    
</div>


    </div>
</div>
</body>

<script src="//cdn.jsdelivr.net/npm/gitalk@1/dist/gitalk.min.js"></script>


<script src="//lib.baomitu.com/jquery/1.8.3/jquery.min.js"></script>
<script src="/js/plugin.js"></script>
<script src="/js/typed.js"></script>
<script src="/js/skyone.min.js"></script>


<link rel="stylesheet" href="/photoswipe/photoswipe.css">
<link rel="stylesheet" href="/photoswipe/default-skin/default-skin.css">


<script src="/photoswipe/photoswipe.min.js"></script>
<script src="/photoswipe/photoswipe-ui-default.min.js"></script>


<!-- Root element of PhotoSwipe. Must have class pswp. -->
<div class="pswp" tabindex="-1" role="dialog" aria-hidden="true">
    <!-- Background of PhotoSwipe. 
         It's a separate element as animating opacity is faster than rgba(). -->
    <div class="pswp__bg"></div>
    <!-- Slides wrapper with overflow:hidden. -->
    <div class="pswp__scroll-wrap">
        <!-- Container that holds slides. 
            PhotoSwipe keeps only 3 of them in the DOM to save memory.
            Don't modify these 3 pswp__item elements, data is added later on. -->
        <div class="pswp__container">
            <div class="pswp__item"></div>
            <div class="pswp__item"></div>
            <div class="pswp__item"></div>
        </div>
        <!-- Default (PhotoSwipeUI_Default) interface on top of sliding area. Can be changed. -->
        <div class="pswp__ui pswp__ui--hidden">
            <div class="pswp__top-bar">
                <!--  Controls are self-explanatory. Order can be changed. -->
                <div class="pswp__counter"></div>
                <button class="pswp__button pswp__button--close" title="Close (Esc)"></button>
                <button class="pswp__button pswp__button--share" title="Share"></button>
                <button class="pswp__button pswp__button--fs" title="Toggle fullscreen"></button>
                <button class="pswp__button pswp__button--zoom" title="Zoom in/out"></button>
                <!-- Preloader demo http://codepen.io/dimsemenov/pen/yyBWoR -->
                <!-- element will get class pswp__preloader--active when preloader is running -->
                <div class="pswp__preloader">
                    <div class="pswp__preloader__icn">
                      <div class="pswp__preloader__cut">
                        <div class="pswp__preloader__donut"></div>
                      </div>
                    </div>
                </div>
            </div>
            <div class="pswp__share-modal pswp__share-modal--hidden pswp__single-tap">
                <div class="pswp__share-tooltip"></div> 
            </div>
            <button class="pswp__button pswp__button--arrow--left" title="Previous (arrow left)">
            </button>
            <button class="pswp__button pswp__button--arrow--right" title="Next (arrow right)">
            </button>
            <div class="pswp__caption">
                <div class="pswp__caption__center"></div>
            </div>
        </div>
    </div>
</div>




</html>
